0 


AD-A275  307 


A 

R 

M 

S 

T 

R 

0 

N 

G 


L 

A 

B 

O 

R 

A 

T 

0 

R 

Y 


AL-SR-1992-  0025 


BRAIN  ACTUATED  CONTROL 
OF  A  ROLL  AXIS  TRACKING  SIMULATOR  (U) 


Andrew  M.  Junker 

Armstrong  Aerospace  Medical 
Research  Laboratory 
Wrmht-Patterson  AFB  OH  45433-6573 


John  H.  Schnurer 
David  F.  Ingle 
Craig  W.  Downey 

Logicon  Technical  Services,  Inc. 
P.O.  Box  317258 
Dayton  OH  45431-7258 


CD 

■£* 

I 

O 

CO 

CD 

00 

00 


Best  Available  Copy 

November  1989 


OTIC 

KjELECTE 
fEB  0  4 1994! 

a 


Final  Report  for  Period 
4  November  1988  to  31  August  1990 

tig  H  Ip  0.b  D  Pi 

Approved  for  public  release;  distribution  is  unlimited. 

94  2  03  165 


AIR  FORCE  SYSTEMS  COMMAND 
WRIGHT-PATTERSON  AIR  FORCE  BASE/  OHIO  45433-6573 


NOTICES 


When  US  Government  drawings,  specifications,  or  other  data  are  used  for  any  purpose  other  than 
a  definitely  related  Government  procurement  operation,  the  Government  thereby  incurs  no 
responsibility  nor  any  obligation  whatsoever,  and  the  fact  that  the  Government  may  have 
formulated,  furnished,  or  in  any  way  supplied  the  said  drawings,  specifications,  or  other  data,  is 
not  to  be  regarded  by  implication  or  otherwise,  as  in  any  manner  licensing  the  holder  or  any  other 
person  or  corporation,  or  conveying  any  rights  or  permission  to  manufacture,  use,  or  sell  any 
patented  invention  that  may  in  any  way  be  related  thereto. 

Please  do  not  request  copies  of  this  report  from  the  Armstrong  Laboratory.  Additional  copies  may 
be  purchased  from: 


National  Technical  Information  Service 
5285  Royal  Road 
Springfield.  Virginia  22161 

Federal  Government  agencies  and  their  contractors  registered  with  the  Defense  Technical 
Information  Center  should  direct  requests  for  copies  of  this  report  to: 

Defense  Technical  Information  Center 
Cameron  Station 
Alexandria.  Virginia  22314 

DISCLAIMER 

This  Special  Report  is  published  as  received 
and  has  not  been  edited  by  the  Technical  Editing 
Staff  of  the  Armstrong  Laboratory. 

TECHNICAL  REVIEW  AND  APPROVAL 


AL-SR- 1992-0025 


This  report  has  been  reviewed  by  the  Office  of  Public  Affairs  (PA)  and  is  releasable  to  the  National 
Technical  Information  Service  (NTIS).  At  NTIS,  it  will  be  available  to  the  general  public, 
including  foreign  nations. 

The  voluntary  informed  consent  of  the  subjects  used  in  this  research  was  obtained  as  required  by 
Air  Force  Regulation  169-3. 


This  technical  report  has  been  reviewed  and  is  approved  for  publication. 


FOR  THE.GOMMANDER 


FTM  D  HOFF  rUlnff 


DUG  QUALITY  INSPECTED 


KENNETH  R.  BOFF,  Chief 


Human  Engineering  Division 
Armstrong  Laboratory 


8 


|  fooosalon  For  j 

HTIS 

HRAkl 

s' 

DTIC 

TAB 

□ 

Waanncunond 

□ 

Just  1 

Pfv 

DiPtrl 

A.TulJ  ciMJ  I  ■  > 

AvGdi  -  ■.!. 

.V  ;• 

L'tst 

Spec!- 

Nov  1989 


Final  Rpt  (4  Nov  88-31  Aug  90) 


C  F33615-85-C-0S41 

3rarn  Actuated  Control  of  a  Roll  Axis  PE  61101F 

Tracking  Simulator  PR  7184 

. TA  14 

Andrew  M.  Junker,  John  H.  Schnurer*,  WU  D7 

David  F.  Ingle*,  Craig  W.  Downey* 


Systems  Research  Laboratories,  Inc. 
2800  Indian  Ripple  Road 
Dayton  OH  4S440 


Armstrong  Aerospace  Medical  Research  Laboratory 

Human  Engineering  Division,  AFSC,  HSD  AL-SR-1992-  0025 

'.-.’r ignt-Patterson  AFB  OH  45433-6573 


Published  in  the  1989  IEEE  International  Conference  on 
Systems,  Man,  and  Cybernetics,  pp  714-717. 


Approved  for  public  release; 
distribution  is  unlimited. 


Investigations  of  steady-state  evoked  electroencephalographic  potentials  as 
indicators  of  mental  workload  have  led  to  successful  implementation  of  a  closed- 
loop  paradigm  allowing  the  possibility  of  brain  actuated  control.  A  three-channel 
Lock-in  Amplifier  System  (LAS-3)  was  used  to  obtain  an  artifact-free  brain 
resonance  signal  at  a  specified  frequency.  Lead-lag  compensation  of  the  LAS-3 
gain  signal  was  investigated  as  a  means  to  produce  the  brain  actuated  control.  A 
Roll  Axis  Tracking  Simulator  (RATS)  was  used  as  a  kinesthetically  salient 
control/feedback  environment.  This  paper  discusses  efforts  to  make  use  of  a 
methodology  for  using  the  derivative  of  selected  brain  resonances  as  a  control 
signal  in  the  kinesthetically  salient  environment  of  the  RATS. 
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ABSTRACT 

Investigations  of  steady-state  evoked 
EEG  potentials  as  indicators  of  mental 
workload  have  "led  to  successful 
implementation  of  a  closed-loop  paradigm 
allowing  the  possibility  of  brain  actuated 
control.  A  three  channel  Lock-in 
Amplifier  System  (LAS-3)  was  used  to 
obtain  an  artifact  free  brain  resonance 
signal  at  a  specified  frequency.  Lead-lag 
compensation  of  the  LAS-3  gain  signal  was 
investigated  as  a  means  to  produce  the 
brain  actuated  control.  A  Roll  Axis 
Tracking  Simulator  (RATS)  was  used  as  a 
kinesthetlcally  salient  control/feedback 
environment.  This  paper  discusses  efforts 
to  make  use  of  a  methodology  for  using  the 
derivative  of  selected  brain  resonances  as 
a  control  signal  in  the  kinesthetlcally 
salient  environment  of  the  RATS. 


1.  I MTBODUCTIOB 

Investigations  of  steady-state  evoked 
potentials  as  indicators  of  mental 
workload  have  led  to  successful 
implementation  of  a  closed-loop  paradigm 
allowing  the  possibility  of  brain  actuated 
control  (Junker  et.  al..  1987).  Hardware 
has  been  developed  which  permits  timely 
and  frequency  specific  feedback  to  human 
operators  of  the  resonances  embedded  in 
their  ongoing  EEG.  Control  of  these 
resonances  has  been  demonstrated  by  all 
subjects  tested.  These  efforts  have  led 
to  an  ongoing  research  program  designed  to 
quantify  mental  resource  allocation  during 
performance  of  the  Criterion  Task  Set 
(Shlngledecker,  1984).  In  addition,  a 
system  which  rejects  broadband  noise  and 
nonspecific  broadband  control  artifacts 
has  been  developed. 


While  engaged  in  the  above  research 
an  interest  to  explore  two  new  questions 
developed.  The  first  addresses  the 
feasibility  of  developing  a  more  sensory 
compelling  feedback  environment.  This 
need  arises  from  observations  that  Brain 
Actuated  Control  (BAC)  success  is 
sensitive  to  the  salient  nature  of  the 
feedback  cues  provided  to  achieve  loop 
closure.  The  more  involving  the  loop 
closure  cues  are.  the  more  effective  the 
training  of  BAC  appears  to  be.  This 
increase  in  effectiveness  should  yield  a 
corresponding  decrease  in  the  training 
time  necessary  to  learn  BAC.  Therefore  it 
is  hypothssized  that  kinesthetic  cuss,  in 
this  case  acceleration  cues  resulting  from 
control  of  a  motion  environment,  would 
enhance  the  learning  of  BAC. 

Secondly,  the  question  of  what  the 
BAC  signal  represents  in  a  control  context 
needs  to  be  addressed.  In  other  words, 
how  should  the  process  of  controlling 
one’s.  EEQ  resonance  be  utilized  for 
machine  control?  Previous  experlmsntatlon 
has  revealed  that  subjects  are  better  at 
controlling  change  of  resonance 
(Increase/decrease)  rather  than  holding 
constant  resonance  levels  (Junker, 

Schnurer  .Ingle.Downey,  1988).  This  mode 
of  control  (l.e.  undershoot  and 
overshoot)  can  be  observed  when  subjects 
are  engaged  in  tracking  tasks  involving 
the  control  of  a  lagging  or  sluggish 
system.  From  our  previous  results  we 
concluded  that  the  BAC  signal  was  lagging 
ths  subject's  'dssired  control*  at  the 
neural  level  resulting  in  the  inability  to 
keep  the  BAC  at  a  constant  level.  This  is 
reasonable  to  hypothesize  since  there  is 
an  inherent  integration  or  filtering 
effect  from  the  scalp  and  the  equipment 
used'  to  produce  the  present  BAC  signal. 
Lead  compensation,  in  the  form  of 
differentiation,  of  the  BAC  signal  should 
reduce  this  problem.  Thus  the  lead 
compensated  EEC  resonance  signal  was  used 
as  the  BAC  and  as  feedback  to  the 

subjects. 
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Figure  2.1  Lock-in  Amplifier  System 


2.  METHODOLOGY 

2.1  Loop-Closure  end  EIQ  Detection 

Apparatus 

Equipment  development  proceeded  along 
the  following  lines.  A  three  channel 
Lock-in  Amplifier  System  (LAS-3),  shown  in 
figure  2.1,  was  created  to  obtain  an 
artifact  free  brain  resonance  signal  at  a 
specified  frequency.  The  LAS-3  is  an 
analog  device  which  provides  near  real¬ 
time  frequency  domain  gain  and  phase 
information  for  a  signal  (EKG  in  this 
case)  at  a  reference  frequency.  Lead-Lag 
compensation  of  the  LAS-3  gain  signal  was 
added  as  a  means  to  produce  a  responsive 
brain  actuated  control  signal. 

Gold  cup  surface  electrodes, 
manufactured  by  Grass,  are  placed  on  the 
scalp  over  the  occipital  cortex  01-02  and 
mastoid  process  for  ground  (according  to 
the  10-20  International  system).  In  order 
to  achieve  good  electrical  contact,  the 
scalp  is  mildly  abraded  by  brisk  rubbing 
with  an  alcohol  soaked  gauze  pad.  A 
nonabrasive  electrode  paste.  Grass  EC2 
Electrode  Creasu  is  put  on  the  cleansed 
area,  with  electrode  affixed  to  the 
subjects'  scalps  with  the  same  electrode 
cream.  No  adverse  reactions  to  these 
procedures  or  materials  have  been 
encountered  in  our  laboratory. 


2.2  Control/Feedback  Environment 

A  Roll  Axis  Tracking  Simulator  (RATS) 
is  used  in  a  klnesthetlcally  salient 
control  paradigm.  Within  the  RATS, 
subjects  employ  a  manually  operated 
actuating  mechanism  as  a  means  of 
•ngaglng/disengaglng  their  brain  actuated 
control.  The  direction  of  the  BAC  input 


is  indicated  with  a  horizontal  light  bar 
(zero  input  referenced  to  center)  and  a 
left/rigbt  audio  feedback  tone.  The 
interior  layout  of  the  RATS  cab  is 
illustrated  in  Figure  2.2.  The  evoking 
stimulus  is  provided  by  two  fluorescent 
tubes  mounted  within  lexan  tubes.  A  video 
monitor  provides  roll  position  and  roll 
velocity  cues.  As  Indicated  in  Figure  2.2 
the  light  bar  and  audio  displays  are 
intended  to  provide  cues  equivalent  to 
those  that  would  normally  be  available 
klnesthetlcally  from  manipulation  of  a 
manual  controller. 


Figure  2.2  Interior  Layout  of  RATS  cabin 


Experimental  Plan 

Closed-loop  tracking  with  vinual 
n i v .  visual  and  kinesthetic,  and 
kinesthetic  only  cuei  is  performed.  In 
each  case  roll  disturbance  regulation  is 
added.  A  digitally  generated  low 
frequency  *uB-of-*me  waves  is  used  as  the 
disturbance  input.  The  resulting  closed- 
loop  block  diagram  for  this  system  is 
illustrated  in  Figure  2.3. 

Data  is  collected  which  is  frequency 
coupled  to  the  sum-of-sines  disturbance. 
This  allows  computation  of  various  system 
transfer  functions.  Performance  scores 
and  system  transfer  functions  provide  a 
basis  for  evaluation  of  the  LAS-3/RATS 
equipment  (bandwidth,  stability)  and  human 
SAC  capabilities. 

Subjects  are  exposed  to  static 
conditions  or  nonhazardous  angular  motion 
on  the  RATS  while  observing  a  CRT  display 
and  performing  BAC  tracking  control. 

During  experimental  sessions  in  the  RATS 
subjects  are  secured  with  a  five  point 
seat  belt  and  shoulder  harness.  The 
experimenter  is  in  continuous  contact  with 
the  subject  at  all  times  through  an 
intercom  which  has  a  live  subject  mike. 

The  experiment  is  divided  into  three 
phases:  Phase  I-Subject  Introduction  to 
BAC  and  RATS  control:  Phase  II  Training 
With  BAC  and  RATS;  and  Phase  III-  Data 
collection  During  BAC  Closed-Loop 
Disturbance  Regulation. 

Phase  I.  The  basis  of  this  phase  is 
to  introduce  subjects  to  the  experimental 
equipment  and  the  concept  of  brain 
actuated  control.  At  first  subjects  have 
the  opportunity  to  control  the  RATS  with  a 
manual  controller  to  become  comfortable 
with  the  motion  capabilities  of  the  RATS. 
Subjects  are  allowed  to  select  a  resonance 


frequency  at  which  they  will  accomplish 
BAC  (between  13Hz  and  20Hz).  At  this  tlo» 
the  experimenter  is  able  to  characterize 
the  subject’s  evoked  EEQ  response.  This 
phase  normally  takes  two  experimental 
sessions  to  complete. 

Phase  II.  The  results  from  the  first 
phase  are  used  to  set  parameters  for  the 
second  phase  in  which  subjects  tram  in 
BAC  on  the  RATS.  Initially,  training 
consists  of  simply  learning  to  control  the 
RATS  with  no  external  disturbance.  AS 
subjects  become  comfortable  with  BAC 
control  a  simplified  disturbance  is  added 
to  the  closed-loop  system.  As  subjects 
become  proficient  at  disturbance 
regulation  using  BAC.  introduction  of  a 
3um-of-sines  disturbance  occurs  marking 
the  beginning  of  Phase  III.  Phase  II 
training  requires  approximately  7 
experimental  sessions  for  each  subject 
tested. 

Phase  III.  This  phase  involves  data 
collection  and  training  with  tha  computer 
generated  sum-of-sines  disturbance  input. 
Data  collection  continues  until 
performance  scores  (RM3  error)  reaah 
consistent  levels.  Three  conditions  are 
tested:  static  tracking  with  visual  cues 
using  BAC.  motion  tracking  with  visual 
cues  using  BAC.  and  motion  tracking  with 
no  visual  cues  using  BAC. 

Data  analysis  consists  of  tabulation 
and  analysis  of  RMS  error  icofm  for 
quantifying  learning  rate*  and 
effectiveness  of  accomplishing  closed-loop 
control.  Frequency  analysis  of 
disturbance  regulation  data  provides 
transfer  functions  of  human  and  machine 
systems.  From  these  transfer  functions 
effectivenss*  of  BAC  control  relative  to 
current  manual  control  is  evaluated. 


Figure  2.3  Closed-loop  RATS/LAS-3  system 


PRELIMINARY  RESULTS  AND  DISCUSSION 


At  present  the  pilot  study  is  in  its 
initial  data  collection  stage.  However, 
even  at  this  preliminary  stage 
participants  subjectively  claim  they  feel 
the  onset  of  control  in  the 
kinesthetically  salient  environment  (RATS) 
at  an  earlier  point  than  subjects  who 
trained  in  a  totally  static  mods.  This, 

1 f  true,  could  be  from  two  possible 
sources.  First,  the  RATS  provides  a 
subject  with  feedback  imperative  of  a  much 
higher  magnitude  than  can  be  found  under 
static  conditions.  Including  a  subject’s 
kinesthetic  sense  in  a  feedback  loop 
allows  an  individual  to  respond  and 
compensate  for  his  dynamic  environment  in 
much  the  same  way  people  learn  to  regain 
their  equilibrium  and  'right'  themselves 
after  stumbling.  The  second  cause  could 
be  the  effectiveneaj  of  lead-lag 
compensation  of  the  physiological  measure 
(specific  frequency  EEQ).  This  lead-lag 
compensation  results  in  a  bandwidth 
incret.j  of  the  measure  such  that  feedback 
of  the  sensed  physiological  signal  comes 
fast  enough  in  terms  of  RATS 
responsiveness. 
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